HIGHLIGHTS OF THE CURRENT ON-TIME PERFORMANCE METHODOLOGY 
Each line is... 

• Randomly sampled once each six months 

• monitored on a minimum of 10 days covering both weekdays and weekends (during each 
sampling period) 

• checked mid-route for at least one hour during each of the following four time periods (when 
applicable): morning rush (6-9a), midday (lOa-lp), evening rush (4-7p), and nighttime (8-llp). 
Expresses and other lines that operate fewer than 19 hours are monitored during the above 
times when they are in service. 

• Monitored in both directions for average of 19 hours per sampling period. Express lines are 
monitored for about six hours, non-evening lines are monitored for about 18 hours and others 
are monitored for an average 24 hours. 

• not mathematically weighted as part of the overall on-time performance calculation (although 
the increased number of observations associated with high frequency lines results in an informal 
weighting of results) 

• manually monitored by traffic checkers using handheld devices which are synched to a 
database used to calculate the quarterly results 



How do we define on-time? 

Sec 8A.103 of the Charter states that "a vehicle is considered on-time if it is no more than one minute 
early or four minutes late (-1/+4) as measured against a published schedule that includes time points. 



What are some of the quirks of the existing methodology? 

• In the late 90s each on-time performance observation was directly correlated with the run 
number of the vehicle observed so the actual time could be checked against the scheduled 
time. This original methodology for measuring technical schedule adherence is the most 
appropriate way to measure the consistency of service delivered. 

• The methodology was modified by Michael Burns. To this date, each on-time performance 
observation is matched to the closest schedule time for the location (regardless of run 
number) to maximize the number of observations falling within the -1/+4 minute on-time 
performance window. 

o This reflects customer observed on-time performance and increases reported on-time 
performance by approximately five to eight percent. Lines and routes with headways 
longer than 10 minutes are minimally affected by the matching method. The biggest 
impact is on lines and routes with headways shorted than seven to eight minutes. 

o Matching increases reported on-time performance by as much as five to eight percent. 



• The software used to tabulate traffic checkers' handheld entries "clips" time observations. As 

a result observations between +4:01 and +4:59 minutes late are treated as +4:00 and thus, are 
considered on time. Clipping enhances reported on time performance by approximately three 
to five percent. 

• Missed runs are not mathematically factored into on-time performance results but resulting 
service gaps affect on-time performance due to longer boarding times associated with crush 
loads. 

o The likely impact of this omission is an increase in reported on-time performance by 
two to four percent. 

• Inbound on-time performance of light rail lines is not tabulated because "service into the 
tunnel cannot be controlled". 

o The likely impact on Systemwide results is less than 1 percent. 

o Individual light rail lines see a much more significant improvement as a result of this 
methodology. 

• All above characteristics lead to a reported on-time performance figure that is 13-18 percent 
higher than actual schedule adherence. 



Has the methodology changed recently? 

• Yes. Starting with FY10 Ql we started reporting ridership-weighted on-time performance as well. 
This generally results in a 1 to 1.5 percent increase in performance. 

What are the pros and cons of moving from manual counting to using Automatic Passenger Counters 
(APCs) and/or NextMuni? 

• Pros 

o NextMuni provides near 85 percent sampling of all time points in the system 

o APCs can provide up to 30 percent sampling for bus (which is still significantly higher 

than manual collection) 
o The near real-time availability of data is much more useful for decision makers than the 

manually collected data 
o Potential labor savings and/or reallocation of existing staff to more value added duties 

• Cons 

o Use of NextMuni or APCs would likely result in a hard-to-explain 10-15 percent drop in 
reporting on-time performance 

■ One recommendation might be to report "technical on-time performance" 
(using raw NextMuni/APC results) and "customer observed on-time 
performance" (using the Michael Burns matching methodology). 

o NextMuni reporting for vehicles in the downtown core and/or arriving at terminals can 
be inconsistent. 

■ However, reporting for vehicles departing terminals and arriving at midpoints is 
good and would provide ample data for on-time performance calculations. 



